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DETAILED ACTION 
Response to Amendment 

1 . This Office Action is responsive to Applicant's remarks received on May 23, 
2008. Claims 1-10, 13, 15-17 and 19-28 remain pending. 

Specification 

2. The abstract of the disclosure is objected to because it contains reference 
numerals from the drawings. Correction is required. See MPEP § 608.01(b). 

Claim Objections 

3. The following is a quotation of 37 CFR 1 .75(a): 

The specification must conclude with a claim particularly pointing out and distinctly claiming the subject 
matter which the applicant regards as his invention or discovery. 

4. Claims 1-4, 7, 8, 13, 19, 22, 27 are objected to under 37 CFR 1 .75(a), as failing 
to particularly point out and distinctly claim the subject matter which application regards 
as his invention or discovery. 
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Claim 1 requires "the class composite image" in line 9. It is unclear whether this 
is intended to be the same as or different from the "three-dimensional class composite 
image" in line 7. The following will be assumed for examination purposes: - the three- 
dimensional class composite image ~. The same applies to claim 2, line 3, claim 3, line 
2, and claim 8, line 2. 

Claim 4 requires a "grid pattern" in line 3. It is unclear whether this is intended to 
be the same as or different from the "three-dimensional grid pattern" in line 8 of claim 1 . 
The following will be assumed for examination purposes: ~ three-dimensional grid 
pattern ~. The same applies for claim 7, line 1 and claim 8, line 1 . 

Claim 13 requires a "three-dimensional class composite image" In line 10. It is 
unclear whether this is intended to be the same as or different from the "three- 
dimensional class composite image" in line 7. The following will be assumed for 
examination purposes: - the three-dimensional class composite image --. 

Claim 19 requires a "grid pattern" in line 10. It is unclear whether this is intended 
to be the same as or different from the "initial grid pattern" in line 8. The following will 
be assumed for examination purposes: ~ the initial grid pattern ~. The same applies for 
line 1 1 , claim 22, line 2, claim 27, line 2. 



Claim Rejections - 35 USC § 103 
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5. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 1-9, 13, 15-17, 19-23 and 25-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Murphey et al. ("Feature Extraction for a Multiple Pattern...", 
IEEE Article), common knowledge in the art as evidenced by Covell et al. (US 
7,003,134) and Krumm (US 5,983,147). 

Regarding claim 1, Murphey et al. discloses a system for selectively generating 
training data for a pattern recognition classifier ("multiple pattern classification neural 
network system" at section 3, line 1) from a plurality of training images representing an 
output class ("k sets of training images Tr^, Tr2,...,Trk, where images in Tn belong to 
class i" at section 2, paragraph 2, line 1), said system comprising: 

a plurality of training images representing an output class ("k sets of training 
images Tm, Tr2,...,Trk, where images in Tn belong to class i" at section 2, paragraph 2, 
line 1); 

an image synthesizer that combines the plurality of training images into a class 
composite image ("generates a feature image for every class of patterns" at page 221 , 
left column, line 5); 
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a grid generator tliat generates a grid pattern representing the output class from 
the class composite image ("refining segment grids in each class feature image" at page 
221, right column, line 9); and 

a feature extractor that extracts feature data from the plurality of training images 
according to the generated grid pattern ("computing the features from subimages 
defined by these grids" at page 221, right column, fourth full paragraph, line 6). 

Murphey et al. does not disclose three-dimensional training images, a three- 
dimensional class composite image and a three-dimensional grid pattern. 

However, it is well-known in the art to utilize three-dimensional images that 
provide depth information for purposes of pattern recognition. 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to utilize three-dimensional information for the training images, 
class composite image and grid pattern as the depth information is "less sensitive to 
illumination and shading effects than intensity data as an object translates and rotates 
through space. Hence, the depth data is frequently more reliable than the brightness 
information" Covell et al. at col. 3, line 25). 

The Murphey et al. and common knowledge combination does not disclose a 
stereo vision system that images the interior of a vehicle to provide the plurality of three- 
dimensional training images. 

Krumm et al. teaches a system for selectively generating training data for a 
pattern recognition classifier ("This system was tested in a vehicle that had been trained 
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with approximately 100 images of eacli situation" at col. 5, line 14) associated with a 
vehicle occupant safety system ("present invention relates generally to automotive 
vehicle occupant restraint systems, particularly air bags" at col. 1, line 14) comprising a 
stereo vision system (figure 1 , numeral 26) that images the interior of a vehicle to 
provide the plurality of three-dimensional training images ("disparity image-one that 
gives disparity values at every point in the image" at col. 6, line 17; "disparity images are 
functions of 3D structure" at col. 7, line 31 ; "disparity training images" at col. 7, line 36). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the vision system of Krumm at al. to provide the training images 
of the Murphey et al. and common knowledge combination as the disparity images "do 
not need the preprocessing normalization and histogram steps utilized on the intensity 
images" (Krumm at col. 7, line 32). 

Regarding claim 2, the Murphey et al., common knowledge and Krumm et al. 
combination discloses a system wherein the grid generator generates the grid pattern 
according to at least one attribute of interest associated with the three-dimensional 
class composite image ("In order to emphasize the areas that have distinct features of 
each class, the dynamic grids finding algorithm uses an iterative procedure that 
increases the resolution of important feature area" Murphey et al. at page 221 , right 
column, line 6). 

Regarding claim 3, the Murphey et al., common knowledge and Krumm et al. 
combination discloses a system wherein the grid pattern divides the three-dimensional 
class composite image with a plurality of intersecting planes and curved surfaces into a 
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plurality of sub-images ("divided the class feature image, say feature_image_cl_i for 
class i, into a coarse scale subimages" at page 221, right column, line 13; a curve can 
be defined as "the intersection of two surfaces in three dimensions" as given in the 
American Heritage Dictionary), the feature extractor extracting data relating to each of 
the plurality of sub-images ("computing the features from subimages defined by these 
grids" at page 221, right column, fourth full paragraph, line 6). 

Regarding claim 4, the Murphey et al., common knowledge and Krumm et al. 
combination discloses a system wherein the grid generator operates according to a grid 
generation algorithm to select one of the plurality of sub-images according to an 
attribute of interest ("iterative procedure that looks for the subimage j that has the 
greatest energy" at page 221, right column, second full paragraph, line 5) and modifies 
the three-dimensional grid pattern according to the selected sub-image ("Once the 
subimage j is found and assume the subimage has size 2'^'X2'^^ then the subimage is 
divided into 4 new subimages" at page 221 , third full paragraph, line 1). 

Regarding claim 5, Murphey et al. discloses a system wherein the attribute of 
interest is a maximum average grayscale value out of a plurality of average grayscale 
values associated with respective sub-images ("average value of each subimage" at 
page 221, right column, fourth full paragraph, line 12; "gray scale intensity images" at 
page 221 , left column; "subimage j that has the greatest energy" at page 221 , right 
column, second full paragraph, line 6). 

Regarding claim 6, the Murphey et al., common knowledge and Krumm et al. 
combintion discloses the elements of claim 4 as described above. 
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The Murphey et al., common knowledge and Krumm et al. combintion does not 
explicitly disclose that the attribute of interest is a maximum grayscale variance out of a 
plurality of grayscale variances associated with the respective sub-images. 

However, as shown in the disclosure of Murphey et al., depending on the input 
image type, an appropriate feature type may employed for optimum results ("For 
different types of training images, one can use feature image differently" at page 221 , 
left column). Furthermore, Murphey et al. discloses the use of average grayscale 
values as the attribute of interest as described above. 

Therefore, choosing maximum grayscale variance as the attribute of interest 
would have been obvious at the time the invention was made to one of ordinary skill in 
the art as the variance of a sample is directly applicable to the average of the sample 
and would better distinguish between types of training images. 

Regarding claim 7, the Murphey et al., common knowledge and Krumm et al. 
combination discloses a system wherein the three-dimensional grid pattern is modified 
as to divide the selected sub-image into a plurality of sub-images ("the subimage is 
divided into 4 new subimages" at page 221 , third full paragraph, line 2). 

Regarding claim 8, the Murphey et al., common knowledge and Krumm et al. 
combination discloses a system wherein the three-dimensional grid pattern is iteratively 
modified until a grid pattern that divides the three-dimensional class composite image 
into a threshold number of sub-images has been generated ("The above process 
repeats until the number of subimages exceeds a preset number" at page 221 , right 
column, third full paragraph, line 3). 
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Regarding claim 9, Murphey et al. discloses a system further comprising a 
pattern recognition classifier that is trained using the extracted feature data ("class grids 
obtained by the above procedure are used to training classifiers" at page 222, left 
column, first paragraph 1, line 1). 

Regarding claim 13, Murphey et al. discloses a system for selectively generating 
training data for a pattern recognition classifier ("multiple pattern classification neural 
network system" at section 3, line 1) associated with a vehicle occupant safety system 
("trained the system to classify occupant inside a vehicle" at section 3, line 3) 
comprising: 

a plurality of training images representing an output class ("k sets of training 
images Tm, Tr2,...,Trk, where images in Tri belong to class i" at section 2, paragraph 2, 
line 1); 

an image synthesizer that combines the plurality of training images to provide a 
class composite image ("generates a feature image for every class of patterns" at page 
221, left column, line 5). 

a grid generator that generates a grid pattern, comprising a plurality of 
intersecting planes and curved surfaces, arranged to provide sub-image regions, 
representing the output class ("refining segment grids in each class feature image" at 
page 221 , right column, line 9; a curve can be defined as "the intersection of two 
surfaces in three dimensions" as given in the American Heritage Dictionary) from the 
class composite image ("class feature image" at page 221, left column, at "step 3"); and 
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a feature extractor that extracts training data from the plurality of training images 
according to the generated grid pattern ("computing the features from subimages 
defined by these grids" at page 221 , right column, fourth full paragraph, line 6). 

Murphey et al. does not disclose three-dimensional training images and a three- 
dimensional class composite image. 

However, it is well-known in the art to utilize three-dimensional images that 
provide depth information for purposes of pattern recognition. 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to utilize three-dimensional information for the training images 
and class composite image as the depth information is "less sensitive to illumination and 
shading effects than intensity data as an object translates and rotates through space. 
Hence, the depth data is frequently more reliable than the brightness information" Covell 
et al. at col. 3, line 25). 

The Murphey et al. and common knowledge combination does not disclose a 
stereo vision system that images the interior of a vehicle to provide the plurality of three- 
dimensional training images. 

Krumm et al. teaches a system for selectively generating training data for a 
pattern recognition classifier ("This system was tested in a vehicle that had been trained 
with approximately 100 images of each situation" at col. 5, line 14) associated with a 
vehicle occupant safety system ("present invention relates generally to automotive 
vehicle occupant restraint systems, particularly air bags" at col. 1 , line 14) comprising a 
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stereo vision system (figure 1 , numeral 26) that images tlie interior of a veliicle to 
provide tlie plurality of three-dimensional training images ("disparity image-one that 
gives disparity values at every point in the image" at col. 6, line 17; "disparity images are 
functions of 3D structure" at col. 7, line 31 ; "disparity training images" at col. 7, line 36). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the vision system of Krumm et al. to provide the training images 
of the Murphey et al. and common knowledge combination as the disparity images "do 
not need the preprocessing normalization and histogram steps utilized on the intensity 
images" (Krumm at col. 7, line 32). 

Regarding claim 15, Murphey et al. discloses a system wherein the plurality of 
training images representing the output class includes images of a human adult seated 
within the vehicle interior ("There were four different classes of patterns, adult, child 
empty seat and rear facing infant seas (Rfis)" at section 3, line 4). 

Regarding claim 16, Murphey et al. discloses a system wherein the plurality of 
training images representing the output class includes images of a rearward facing 
infant seat positioned within the vehicle interior ("There were four different classes of 
patterns, adult, child empty seat and rear facing infant seas (Rfis)" at section 3, line 4). 

Regarding claim 17, Krumm et al. discloses a system wherein the plurality of 
training images representing the output class includes images of a human head 
("classified into three classes (empty, RFIS, occupied)" at col. 7, line 9; "various features 
on the occupant (e.g. top of head, shoulders, seat)" at col. 9, line 59). 



Application/Control Number: 10/772,655 Page 12 

Art Unit: 2624 

Regarding claim 19, Murplney et al. discloses a metliod for selectively generating 
training data for a pattern recognition classifier ("multiple pattern classification neural 
network system" at section 3, line 1 , which performs the method) from a plurality of 
training images representing a desired output class ("k sets of training images Tm, 
Tr2,---,Trk, where images in Tn belong to class i" at section 2, paragraph 2, line 1), said 
method comprising the steps of: 

generating a representative image that represents the output class ("generates a 
feature image for every class of patterns" at page 221 , left column, line 5); 

dividing the representative image according to an initial grid pattern to form a 
plurality of sub-images ("divided the class feature image, say feature_image_cl_i for 
class i, into a coarse scale subimages" at page 221 , right column, line 13); 

identifying at least one sub-image formed by said initial grid pattern having at 
least one attribute of interest ("iterative procedure that looks for the subimage j that has 
the greatest energy" at page 221, right column, second full paragraph, line 5); 

modifying said initial grid pattern to add at least one of a plane and a curved 
surface to the initial grid pattern in response to the identified at least one sub-image 
having said at least one attribute of interest so as to form a modified grid pattern ("Once 
the subimage j is found and assume the subimage has size 2^^X2''^ then the subimage 
is divided into 4 new subimages" at page 221 , third full paragraph, line 1 ); and 

using the modified grid pattern to extract respective feature vectors from the 
plurality of training images ("computing the features from subimages defined by these 
grids" at page 221 , right column, fourth full paragraph, line 6). 
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Murphey et al. does not disclose a representative three-dimensional image and 
three-dimensional grid pattern. 

However, it is well-known in the art to utilize three-dimensional images that 
provide depth information for purposes of pattern recognition. 

Therefore, it would have been obvious at the time the invention was made to one 
of ordinary skill in the art to utilize three-dimensional information for the representative 
image and grid pattern as the depth information is "less sensitive to illumination and 
shading effects than intensity data as an object translates and rotates through space. 
Hence, the depth data is frequently more reliable than the brightness information" Covell 
et al.at col. 3, line 25). 

The Murphey et al. and common knowledge combination does not disclose that 
the representative three-dimensional image is generated at a stereo camera. 

Krumm et al. teaches a system for selectively generating training data for a 
pattern recognition classifier ("This system was tested in a vehicle that had been trained 
with approximately 100 images of each situation" at col. 5, line 14) from a plurality of 
training images representing a desired output class ("disparity training images" at col. 7, 
line 36) comprising generating a representative three-dimensional image that 
represents the output class at a stereo camera ("disparity image-one that gives 
disparity values at every point in the image" at col. 6, line 17; "disparity images are 
functions of 3D structure" at col. 7, line 31). 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the vision system of Krumm et al. to provide the training images 
of the Murphey et al. and common knowledge combination as the disparity images "do 
not need the preprocessing normalization and histogram steps utilized on the intensity 
images" (Krumm at col. 7, line 32). 

Regarding claim 20, Murphey et al. discloses a method wherein the step of 
generating a representative image includes combining the plurality of training images to 
form a class representative image class ("generates a feature image for every class of 
patterns" at page 221 , left column, line 5). 

Regarding claim 21, Murphey et al. discloses a method where the step of 
generating a representative image includes averaging grayscale values across 
corresponding pixels in the plurality of training images ("average value of each 
subimage" at page 221 , right column, fourth full paragraph, line 12; "gray scale intensity 
images" at page 221 , left column). 

Regarding claim 22, Murphey et al. discloses a method wherein the step of 
modifying the initial grid pattern includes modifying the initial grid pattern to divide the 
identified sub-images into respective pluralities of sub-images ("the subimage is divided 
into 4 new subimages" at page 221, third full paragraph, line 2). 

Regarding claim 23, Murphey et al. discloses a method wherein the at least one 
attribute of interest includes an average grayscale value associated with a sub-image 
that exceeds a threshold value ("iterative procedure that looks for the subimage j that 
has the greatest energy" at page 221, right column, second full paragraph, line 5; 
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"average value of each subimage" at page 221 , right column, fourth full paragraph, line 
12; "gray scale intensity images" at page 221, left column). 

Regarding claim 25, Murphey at al. discloses a method wherein the at least one 
attribute of interest includes a maximum average grayscale value out of a plurality of 
average grayscale values associated with respective sub-images ("average value of 
each subimage" at page 221, right column, fourth full paragraph, line 12; "gray scale 
intensity images" at page 221 , left column; "subimage j that has the greatest energy" at 
page 221, right column, second full paragraph, line 6). 

Regarding claim 26, Murphey et al. discloses a method wherein the step of using 
the modified grid pattern to extract respective feature vectors from the plurality of 
training images includes applying the modified grid pattern to a training image to form a 
plurality of sub-images from the training image ("divided the class feature image, say 
featurejmage clj for class i, into a coarse scale subimages" at page 221 , right 
column, line 13) and extracting at least one element associated with a respective 
feature vector from each of the plurality of sub-images ("feature vector orf an image in 
either training data or test data is formed by superimposing the grids to the image and 
then computing the features from subimages defined by these grids" at page 221, right 
column, fourth full paragraph, line 3). 

Regarding claim 27, Murphey et al. discloses a method wherein the steps of 
identifying at least one sub-image and modifying the initial grid pattern in response to 
the identified sub-image are repeated iteratively until a termination event is recorded 
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("The above process repeats until tine number of subimages exceeds a preset number" 
at page 221, rigint column, third full paragraph, line 3). 

Regarding claim 28, Murphey at al. discloses a method wherein the termination 
event comprises producing a modified grid that divides the class composite image into a 
threshold number of sub-images ("The above process repeats until the number of 
subimages exceeds a preset number" at page 221, right column, third full paragraph, 
line 3). 

7. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Murphey et al, common knowledge and Krumm et al. as applied to claim 
9 above, and further in view of Gokturk et al. (US 2003/0169906). 

The Murphey et al., common knowledge and Krumm et al. combination discloses 
the elements of claim 9 as described above. 

The Murphey et al., common knowledge and Krumm et al. combination does not 
disclose that the pattern recognition classifier includes a support vector machine. 

Gokturk et al. teaches a system in the same field of endeavor of vehicle occupant 
detection ("One application where occupant classification system is gaining use is with 
vehicle restraint and airbag deployment systems" at paragraph 0060, line 1) wherein the 
pattern recognition classifier includes a support vector machine (figure 1, numeral 120; 
"SVMs" at paragraph 0106, line 7). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to utilize the support vector machine of Gokturk et al. in the classifier of 
the Murphey et al., common knowledge and Krumm et al. combination as "SVMs 
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mininnize the risk of misclassifying previously unseen data" (Gokturk et al. at paragrapli 
0089, line 1) and "Techniques such as SVMs, neural networks and HMMs... could be 
combined by the depth images", thereby yielding a more accurate detection (Gokturk et 
al. at paragraph 0106, line 6). 

8. Claim 24 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Murphey et al, common knowledge and Krumm et al. as applied to claim 
19 above, and further in view of common knowledge in the art as evidenced by Jackway 
et al.(US 2002/0051571). 

The Murphey et al., common knowledge and Krumm et al. combination discloses 
the elements of claim 19 as described above. 

The Murphey et al., common knowledge and Krumm et al. combination does not 
explicitly disclose that the at least one feature value includes a coarseness measure 
associated with each sub-image. 

However, as shown in the disclosure, depending on the input image type, an 
appropriate feature type may be employed for optimum results ("For different types of 
training images, one can use feature image differently" at page 221, left column). The 
method of Murphey et al. discloses the use of texture images as a possible type of 
training image ("texture image" at section 2, paragraph 2, line 4). 

Accordingly, it would have been obvious at the time the invention was made of 
one of ordinary skill in the art to use coarseness as a feature value when evaluating 
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texture images as "Image texture can be qualitatively evaluated as having one or more 
of the properties of fineness, coarseness" (Jackway et al. at paragraph 0004, line 6). 



Response to Arguments 

Summary of Remarks (@ response page labeled 9): The abstract is not 
prohibited from incorporating reference numerals. 

Examiner's Response: MPEP § 608.01(b) section C states "The sheet or sheets 
presenting the abstract may not include other parts of the application or other material" 
as the abstract should enable the reader to " determine quickly from a cursory 
inspection of the nature and gist of the technical disclosure" (see section A). Therefore, 
the objection to the abstract is maintained. 

Summary of Remarks (@ response pages labeled 10 and 11): The Murphey 
reference does not disclose how it would be used for head tracking with regard to 
Owechko. 

Examiner's Response: This argument is moot in view of the new grounds of 
rejection. 
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Sumnnary of Remarks (@ response page labeled 11): The Murphey reference 
does not suggest "any utility to be gained from the addition of the depth data or suggest 
the extension of the grid generation algorithm to depth data". 

Examiner's Response: The above modified rejections incorporate common 
knowledge in the art that explain why one would seek to extend the two-dimensional 
algorithm of the Murphey reference to three dimensions by means of including depth 
data. 



Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, liowever, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to KATRINA FUJITA whose telephone number is 
(571 )270-1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8- 
4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Bali can be reached on (571) 272-7415. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Katrina Fujita/ 
Examiner, Art Unit 2624 



/Vikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



